This invention relates to stereoscopic imaging systems for creating a three-dimensional illusion using motion pictures or video recorders. It is more closely related to autostereoscopic systems, which produce true three-dimensional images which do not require glasses or parallax barriers to create a three-dimensional illusion.
Humans perceive movement in motion pictures and television because of the brain mechanisms underlying such established psychological facts as persistence of vision and the phiphenomenon. Depth is perceived by the interpretation of disparity information from the eyes through a process called stereopsis. What stereopsis is and how it is performed are still a matter of some debate.
Humans have binocular (stereoscopic) vision-two eyes that look in the same direction and whose visual fields overlap. The eyes are horizontally aligned and separated by an interocular distance averaging about 65 mm. Each eye views the scene from a slightly different angle. The scene viewed is focused by the eye's lens onto the retina as a two-dimensional image. The two-dimensional images from each eye are transmitted along the optic nerves to the brain's visual cortex. The monocular and parallax depth information from the eyes is compared and interpreted through stereopsis, to form a true three-dimensional view.
A distinction must be made between monocular depth cues and parallax information in the visual information received. Both eyes provide essentially the same monocular depth cues, but each provides different parallax depth information, a difference that is essential to produce a true three-dimensional view.
It is possible to perceive depth to a certain extent in a two-dimensional image. Monocular depth is perceived when viewing a still photograph, a painting, or standard television and movies, or when looking at a scene with one eye closed. It is perceived without the benefit of binocular parallax depth information. Such depth relations are interpreted by the brain from monocular depth cues such as relative size, overlapping, perspective, and shading.
Even though human eyes are horizontally aligned, the brain will process parallax information from any direction. It has been reported that vertical parallax information when displayed at a rate of 4 to 30 Hz, produces a sense of depth that is superior to that produced by horizontal parallax presented in the same manner.
It has also been reported that the fusion range of stereoscopic vision is within a 40 minutes (0.66.degree.) angle for horizontal disparity and up to a 7 minutes (0.1166.degree.) angle for vertical disparity.
Parallax information does not have to be presented to the brain simultaneously. The left and right eye depth information can be presented alternately to the left and right eyes, resulting in depth perception as long as the time interval does not exceed 100 milliseconds. The brain can extract parallax information from a three-dimensional scene even when the eyes are alternately covered and uncovered for periods of up to 100 milliseconds each. The brain can also accept and process parallax information presented to both eyes if sequenced properly. The ideal view cycle sequencing rate is between 3-6 Hz.
True three-dimensional image displays can be divided into two main categories, stereoscopic or binocular and autostereoscopic. Stereoscopic techniques (including stereoscopes, polarization, anaglyphic, Pulfrich, and shuttering technologies) require the viewer to wear a viewing apparatus. Autostereoscopic techniques (such as holography, lenticular screens, parallax barriers, alternating pairs, and parallax scans) produce images with a true three-dimensional illusion without the use of glasses.
Prior art three-dimensional television or motion picture display system, that did not require viewing glasses, alternately displayed views of a scene recorded by two cameras at their respective points of view. U.S. Pat. No. 4,006,291 to Imsand, U.S. Pat. Nos. 4,303,316 and 4,420,230 to McElveen, and U.S. Pat. No. 4,429,328 to Jones, et al describe methods using horizontally, vertically and a combination of horizontally and vertically displaced views. The images produced using the method of Jones, et al did appear three-dimensional, but were extremely unstable and possessed a distracting rocking motion. Jones, in U.S. Pat. No. 4,528,587, attempted to control the rocking motion by using a video mixing device, which intermittently superimposed the second camera's image onto that of the first, rather than alternating images as before. This mixing technique did little to control rocking and resulted in intermittent image softening.
The applicants have experimented with the known alternating-camera methods and concluded that stable three-dimensional images could not be achieved simply by aligning two cameras vertically, horizontally or diagonally and switching between or mixing them at a 4 to 5 Hz view cycle rate. Commercial production standards today are much too high for the image instability and/or softening inherent in these methods.
Unlike stereoscopic techniques, which provide each eye with a different image, alternating techniques provide the same image to both eyes. With a stereoscopic system the brain will compensate for some mismatch of camera lenses, color and luminance differences, and differences in parallax angles. In alternating systems the slightest mismatch is readily perceived.
Image instability (rocking) is caused by a variety of factors. The main cause is the presentation of alternating points of view that differ in parallax and are not in tune to be perceived as depth rather than motion. Since all prior art alternating techniques use two cameras, factors such as improper alignment of cameras, lenses mismatched in focal length and/or focus, chrominance and luminance mismatches, poor quality optics, s- and p- polarization differences, and misplaced convergent point all contribute to image instability. Another problem is the methods used to obtain the parallax information. Provisions must be made for constant parallax and convergent corrections during shooting in order to keep the depth information in tune with the human brain.
Image instability can be rendered less noticeable by the use of masking techniques. Camera motion is very effective in hiding rocking, apparently because the brain places less importance on the rocking motion than on the camera motion. This result could represent some sort of natural stabilizing phenomenon or mechanism of the brain that helps us see clearly when we walk or run, when the images would otherwise bounce.
Proper camera convergence and parallax angle adjustment are also very important. Our tests have shown that if the convergent point is set on the closest object to the camera or closer and the parallax angles are in tune with the scene being shot, the brain tends to disregard background motion, if it is combined with camera motion. If the convergent point is set behind the closest object, that object will rock and the rocking cannot be masked by camera motion or parallax tuning. If the camera moves, the closest object moves, or something enters the frame closer than the convergent point, the convergent point must be pulled back and the parallax angle adjusted (tuned) accordingly. The reverse is also true, if the closet object moves farther away from the camera, the parallax angle should also be adjusted.
The methods and camera system described in U.S. Pat. Nos. 4,815,819 and 4,966,436 to Mayhew and Pritchard require careful camera alignment to eliminate unwanted movement in all depth planes. Precision matching of chrominance and luminance between cameras, and a good deal of operator skill to manipulate disparity, convergence, and time-displacement rates required to maintain a stable image.
Even though this two camera system can deliver a very stable, broadcast-quality video image, it is not ideal for day to day television production. The cameras require constant alignment adjustment. Because of the folded optical path, lenses with a view wider than that of a 32 mm lens can not be used, and zoom lenses are not practical. The fact that the system uses a special mount to hold two cameras and a folded optical path makes it large and heavy.
For all of the reasons above and others the autostereoscopic methods using a single camera are the subject of the applicants U.S. patent application Ser. No. 425,232 filed Oct. 23, 1989, now U.S. Pat. No. 5,014,126 were developed. The methods described in said U.S. patent application do not suffer from any of the matching, alignment, and lens limitations of the prior art. Other single camera systems have been suggested and some even developed for three-dimensional imaging, but all use two lenses or some type of beamsplitter to provide two differing parallax views. The disclosures of all of the applicants aforesaid issued or pending U.S. Patents are included herein by specific reference.
Most prior art shuttering stereoscopic and autostereoscopic motion picture and television techniques use square wave switching methods to alternate between the two points of view, or origin. The abrupt shift in parallax in square wave switching contributes to image instability.
The present application approach is to give each frame its own parallax scan. Each frame and its scan preferably will fall on one or the other side of the nominal optical axis of the camera, which is the point of zero amplitude of the sine wave. The camera imaging plane's optical axis sweeps across the nominal and through positions having parallax.
A parallax scan is different from the prior art point of view or origin. A typical point of view has the same angle of parallax at the start and end of a particular frame's exposure. The angle in radians is determined by the disparity of the point of origin (distance from the nominal) divided by the distance to the point of convergence (for angle in degrees multiply by 57.2958). One point is on one side of the nominal and one the other. Each point may have several frames exposed from it, or as few as one field in video.
Parallax scanning techniques employ a continuously moving imaging plane. A particular frame will start its exposure at one angle of parallax and end it at another angle of parallax, which is greater or less than the starting angle depending on where the frame lies in the scan. A parallax scan can sweep back and forth across the nominal zero point in any direction-horizontal, diagonal or vertical. The scanning motion blurs the background of the frame slightly and therefore helps mask unwanted rocking. The optical axis of the parallax scan is centered on the point of convergence. A parallax scan can achieve a very large angle of parallax in its extreme exposure frame and a high overall average angle of parallax. Because the differences are slight, a sine wave can also be approximated by a parabolic sequence.
It is an object of this invention to provide a recording system for producing the scanning motion which produces the time-shared imagery.
Another object of this invention is to provide a system for moving the scanning components without introducing reaction forces (vibration) in the recorder and its supporting members.
Still another object of this invention is to provide inexpensive scanning techniques suitable for less demanding situations.
An even further object of this invention is to provide a disparity control system which is locked to the camera frames or fields.
A still further object of this invention is to provide a disparity control responsive to the scene velocity.
Another object of this invention is to provide automatic adjustment of the scanning convergent point.
These and other objects are achieved by providing a autostereoscopic image recorder having a single recorder for recording images and including a single optical path through a convergent point between the scene and the recorder, a scanning path structure and a driver for substantially continuously moving the single optical path along a scanning path for a plurality of scanning cycles. The path defining structure may include a rail transverse to the optical path and a second rail displaced from the optical path at an angle to the first rail. The recorder is mounted to move on the rails. The convergent point of the system may be adjusted by moving the recorder and the rail system relative to each other. This may be achieved by moving the rail structure along the optical path or changing the angle of the second or convergent rail. A path defining structure may define a linear path orthogonal to the optical path and include a lens which converges the optical path on the convergent point. Alternatively, the path defining structure may define an arc which is centered about a fixed point in the optical point and the lens for causing the optical path to intersect the second fixed point. The convergent point is one of the first or second fixed points. In either system, the convergent point may be adjusted by changing the distance between the lens and the recorder or the first fixed point.
The driving device which produces vibration-free scanning motion includes a support to which a first and second mass are movably mounted. The recorder is connected to the first mass. A drive for the masses substantially continuously moves the first and second masses in opposite directions such that the recorder moves along the scanning path for a plurality of scanning cycles. The second mass has substantially the same mass as the first mass plus the mass of the recorder. The drive for the masses includes an armature and a stator mounted to the first and second masses. The first and second masses are mounted to the support by a pulley structure to produce the equal and opposite motion.
The convergent point may be adjusted automatically by a device which determines the distance between the recorder and a desired convergent point thereby adjusting the scanning path to maintain the optical path on the desired convergent point during scanning.
The extent or amplitude of the scanning path is adjusted depending upon the degree of motion in the scene or of the apparatus. Because of observed masking techniques, the amount of travel is increased for the degree of motion. The amount of motion is determined by correlation between successive recorded images.
The driver structure is synchronized to the recorder operation. The synchronization controls the drivers for bi-directional operation of the recorders such that double exposure can be produced while maintaining the three-dimensional effect.
In addition to the parallax effect produced by recording a plurality of scanning images during a cycle, additional external stimuli is provided. The position of the recorder along the scanning path is determined and the ultimate image is adjusted. The image may be adjusted within the recording frame or during the display of the recording frame.
Other objects, advantages and novel features of the present invention will become apparent from the following detailed description of the invention when considered in conjunction with the accompanying drawings.